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Evolutionary game dynamics in finite populations is typically subject to noise, inducing 
effects which are not present in deterministic systems, including fixation and extinction. 
In the first part of this paper we investigate the phenomenon of drift reversal in finite 
populations, taking into account that drift is a local quantity in strategy space. Secondly, 
we study a simple imitation dynamics, and show that it can lead to fixation at internal 
mixed-strategy fixed points even in finite populations. Imitation in infinite populations is 
adequately described by conventional replicator dynamics, and these equations are known 
to have internal fixed points. Internal absorption in finite populations on the other hand 
is a novel dynamic phenomenon. Due to an outward drift in finite populations this type of 
dynamic arrest is not found in other commonly studied microscopic dynamics, not even in 
those with the same deterministic replicator limit as imitation. 
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I. INTRODUCTION 

Much of the current interest in dynamic phenomena in the frequency-dependent selection in pop- 
ulation focuses on the differences between stochastic descriptions of evolutionary processes and their 
deterministic counterparts. Traditionally deterministic formulations have been more common in the 
biological literature, presumably due to the fact that the theory of non- linear deterministic differential 
equations is relatively well developed. Aspects of stochastic dynamics that have been studied in recent 
years include (i) probabilities with which mutants invade existing populations [Now ak 2006 , (ii) fixa- 
tion probabilities and time-to-fixation in finite populations [Nowa k~2006l lAltrock Traulse n 2009a] , 
(iii) the differences between dynamical stationary states of deterministic and stochastic processes 
[Traulsen et al 20061 iTrauLscn &: Hauert 2009] IMo bilia 2010] and (iv) the dynamics of pattern for- 
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mation induced by noise [Reiche nbach et al 2007] . 

In the present paper we will focus on two dynamic features of stochastic evolutionary dy- 
namics, namely those of (a) drift reversal and (b) fixation to internal points in strategy 
space. Drift reversal here refers to a phenomenon reported in [Traulsen et al 2005| IClausse n 2007|, 
IClaussen &: Traulsen 2008] , and characterises the observation that some stochastic evolutionary sys- 
tems tend to approach the boundaries of the strategy space at small population sizes, whereas a 
drift in the other direction, towards the centre of strategy space, occurs at larger sizes. In partic- 
ular expressions for a critical population size, at which this reversal of drift occurs, were derived in 
[Traulsen et al 20051 IClaussen 2007| IClaussen k, Traulsen 2008] . It is important to stress that the 
meaning of the word 'drift' is here different from its usual use for example in the context of 'random 
drift', 'genetic drift' or 'neutral drift' [Kimura 19831 IKimura fc Crow 19641 IHubbell 2001] . Instead it 
measures a tendency of the stochastic system to approach an internal fixed point, or to approach the 
boundaries of strategy space respectively, this will become clear below, when we give a more precise 
mathematical characterisation. 

Secondly we analyse the dynamics of a strict proportional imitation process in finite populations, 
the details of this dynamics will be described below. Game dynamical processes commonly described 
in the literature are typically of the following type: in a first step two individuals are drawn at random 
from the population. Secondly, if the two individuals are of distinct types, one individual adopts the 
strategy of the other with a certain rate, typically dependent on the fitnesses of the two individuals, 
and in some cases also on the mean fitness of all individuals in the population. This adaptation 
step is equivalent to the death of one individual, and the birth of another. This type of dynamics 
leads to two distinct sources of stochasticity, one coming from the selection of the two individuals, 
and a second from the randomness of the birth and death processes that occur subsequently. In 
[Altrock &; Traulsen 2009b] an interesting limiting case was introduced and studied. Both types of 
randomness were eliminated, leading to an entirely deterministic process even in finite populations. 
Here we take a different route, and make the birth-death component of the update deterministic in 
that only transition towards strictly more successful strategies are allowed. The randomness in the 
selection of individuals for potential update remains however, and consequently the overall dynamics 
is still stochastic, but as we will see novel phenomena can occur, in particular the possibility of an 
absorbing fixed point in the interior of the strategy space. 

The paper is organised as follows: In Sec. Ill] we will introduce the required definitions and the 



general notation. In Sec. Ill we then analyse the phenomenon of drift reversal in detail, and provide a 
refinement of the findings of [Traulsen et al 2005] IClaussen 20071 IClaussen fc Traulsen 2008] . These 
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are derived for the one-dimensional case of symmetric games with two strategies, but can be gener- 



alised to higher-dimensional dynamics. In Sec. IV we then introduce the strict imitation dynamics, 



and show how it leads to internal absorption in the context of stochastic one-dimensional dynamics. 



Sections [V] and VI and then focus on similar dynamics for cyclic games with three strategies per 



player and for two-population games. In the final section we summarise our findings. 

II. GENERAL DEFINITIONS 
A. Stochastic dynamics in finite populations 

We will consider populations of N agents, and restrict the analysis to processes conserving the size 
of the population. Each individual can be of one of S types (the different species or pure strategies), 
so that the state of the system is characterised by the vector n = (ni, . . . ,ns), where indicates 
the number of individuals of type i G {1, . . . , S}. One always has Yli=i n i = N. For one-population 
models we will label the payoff matrix elements of the underlying game as a^, where i, j G {1, . . . , S}. 
The quantity a^- thus denotes the payoff a player of type % receives in interaction with a player of 
type j. Frequency-dependent fitnesses are then given by 

where we have excluded interaction of one individual with itself. To specify a population dynamical 
process it is then sufficient to define the 'conversion' rates Ti-^j, corresponding to events in which a 
player of type i is replaced by one of type j. We will limit the discussion to processes of the general 
form 

Ti -* = lfN 9ij ®> i^j, (2) 

where /j = 1 — w + wtti and f = (/i, . . . , fs). The model parameter w here denotes the strength of 
selection and takes values w G [0, 1]. The {/;} are often referred to as reproductive fitnesses. In this 
paper we will mostly consider pairwise comparison processes, and the precise choice of w will often be 
unimportant, amounting essentially to choosing a time scale. The form ([2]) is found by, at each time 
step, selecting two players from the population (with replacement), one for potential reproduction 
and one for potential removal. The player selected for potential removal is assumed to be of type i, 
the one for potential reproduction of type j. For a given pair of selected players reproduction and 
death actually only occur at a rate proportional to 5ij(f). The time evolution of the probability 
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density P(n, t) of finding the system in state n at time t is then given by 




= J2(E i ET 1 -l)[T i ^ j (n)P(n,t)}, 



(3) 



where Ei is a shift operator, acting on functions of n by increasing rij by one: Eiip{ni, . . . ,ns) = 
ip(ni, . . . , rti_x,nj + 1, n^+i, . . . , ns). E~ l is the inverse operator, reducing rii by one. It is here 
important to note that the system only has 5 — 1 degrees of freedom, given that Y2i=i n i = N. For 
reasons of notational compactness the above master equation ([3]) has been formulated for the full 
system n = (m, . . . ,ns). This equation will be the starting point for our further investigations of 
stochastic game dynamics in finite populations. 



A description of evolutionary processes for infinite populations, A?" — > oo, is given by the following 
replicator dynamics 



Here Xi = limjv->oo j§ is the fraction of players of type i in the limiting infinite population, and 4>' 
the average fitness in the population 



It is however important to note that the above replicator equation provides an accurate description 
of the dynamics in the limit N — > oo only for a suitable choice of the microscopic dynamics, in 
particular of the functions defined above. This will be explained in more detail below, see also 
[Traulsen et al 20051 IBladon e t al 2010] for further discussion. Throughout this paper the superscript 
oo will indicate that we are referring to quantities defined for infinite populations. These quantities 
will typically be functions of x = (x±, . . . , xg), and can be obtained by a simple replacement of 
limAr_ s . 00 rii/N by x^, in the corresponding quantities for finite populations. For example we have 



B. Deterministic replicator dynamics 




(4) 




(5) 



/f°( x ) = l-w + w^2 j a ij x j . 



III. REVERSAL OF LOCAL DRIFT IN 2 x 2 SYMMETRIC GAMES 



A. 



Definitions 



We will first consider the example of a two-player symmetric game with two strategies, which 
we will label i £ {A,B}. In particular we will address games for which the deterministic replicator 
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dynamics has a stable internal fixed point. Specifically we will choose the so-called Hawk-Dove game, 
also referred to as the game of chicken, defined by the payoff matrix [Nowak 2006] 



(6) 



(6-c)/2 6 
6/2 

with model parameters b, c > 0, to be specified below. Here S = 2 and the state of the population 
is fully specified by the number of individuals carrying strategy A. If the population consists of n 
players of type A, and N — n players of type B one then has 

n — lb — c N — n 

. N - n - 1 b 
vrB(n) = — -■ (7) 

The number n* of individuals of type A at which the fitnesses of the two species are equal, 7r^(n*) = 
7r B (n*) is given 

b N + l, (8) 



n 



c 



if this expression is integer- valued (otherwise there is not such point). We will in the following 
specifically choose 6=1 and c = 2, so that n* = N/2 + 1. This expression is integer-valued whenever 
the total number of individuals in the population, N, is even, and we will restrict the discussion to 
such cases in the following. 

In order to discuss the phenomenon of local drift reversal we will focus on what is referred 
to as the so-called 'local' update process in the literature [Traulsen et al 2605[ IClaussen 2007} 
IClaussen fc Traulsen 2008] , This process is defined by the following transition rates 

rp+f ^-rr i \ n N-n fl w 7r A (n) - ir B (n) \ 

T („) = TWn) = -— + - ^— j , (9) 
rp-t ^- rp / x n N-n fl w ir A (n) - ir B (n) \ 

T („) = T^ B (n) = - — j . (10) 

In this two-strategy case we write T + as a short-hand for T b ^a and T~ for the transition rate in the 
opposite direction. A7r max is an additional model parameter, introduced to ensure transition rates 
remain positive. Specifically this model parameter denotes the maximally possible difference in fitness 
between the fittest and the least fit species [Traulsen et al 20"05] . Given the choices 6 = 1, c = 2 we 
will here use Att = w = 1. 

The deterministic dynamics corresponding to the local update rule, valid in the limit of infinite 
populations, N — > oo can be obtained from these definitions for example from the lowest order terms 
of an expansion of the master equation ^ in the square root of the inverse system size. We will 
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not present the mathematical details here, as these have been discussed at length in the literature 
[Bladon et al 20101 ITTaulsen et al 20051 |van Kampen 1992]. The lowest-order result is obtained as 



7/1 

± = T°°' + (x) - T°°>-(x) = — x(l - x)(l - 2x), (11) 

2A7T max 

where x is the fraction of players of type A in the limit of infinite populations. It is obvious that this 
equation admits an internal fixed point at x* = 1/2 in addition to the trivial fixed points x* = and 
1. Up to a re-scaling of time equation (11) is a specific example of a replicator equation Q, as 



x 

some elementary algebra shows 



B. Inversion of local drift 

The notion of drift in [ Traulsen et al 20051 IClaussen 20071 IClaussen fc Traulsen 2008) refers to 
the tendency of the stochastic dynamics to evolve towards an internal fixed point, or away from it. 
More formally, given a measure of distance of the one-dimensional Hawk-Dove system from the fixed 
point, D°° : [0, 1] — > Rq~, we will ask whether this distance is more likely to increase than to decrease 
if the system is started at a certain point in phase space, or whether there is a higher probability for 
a reduction in distance than there is for an increase of distance. D°° may here for example be the 
Euclidean distance D°°(x) = (x — 1/2) 2 from the internal fixed point x* = 1/2 of the deterministic 
dynamics, but for purposes of general considerations we will leave D°°{-) unspecified until a later 
point. Recall that following our above remarks we will use the notation D°°{x) to refer to the case of 
infinite populations, and D{n) = D°°(n/N) for N finite. The expected change A(n) of the distance 
of the system from the fixed point, conditioned on a starting point x = n/N, is then given by 



A(n) = T + (n)D{n + 1) + T~D(n- 1) - (T+(n) + T~(n)D{ 



n 



_ r+ W -r-(„ ) (fl0O))(i) + ^("Hft-V n,) + 0(A ,-3), (12) 

where (D°°)'{x) and {D°°)"{x) refer to the first and second derivatives with respect to x of the 
function D°°(x). Focus sing now on the Euclidean distance D°°(x) = (x — 1/2) 2 we then have 

AW -£W^(-_A + ZW3!!>. (13) 



For the local process as specified above we find 

2(71-1} 



.. N n(N-n) 1 
A(n) = 



N 2 (N-l) 



1 



n 1\ n{N — n) 1 



N 

This result is illustrated and compared with numerical simulations in Fig. [TJ Simulations are of the 
microscopic process are here carried out using the popular Gillespie algorithm [Gillespie 1976]. As 
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FIG. 1: (Color online) Local drift in the Hawk-Dove game. Lines are from theory (N = 50,100,200 from 
top to bottom at the maximum), symbols from simulations (averaged over 10 6 independent runs). Inset is a 
magnification of the central peak, where an outward drift is observed. 



seen in the figure the theoretical result (13) is confirmed by these computer experiments. Results 
show that for any fixed system size N, a local outward drift, A > occurs for starting points, 
x = n/N, close to the deterministic fixed point. If the starting point is sufficiently far away from the 
fixed point an inward drift occurs instead. As demonstrated in the inset of Fig. [l]the region in which 
an outward drift occurs shrinks consistently as the system size is increased. This is illustrated further 
in Fig. [2j where we show where in strategy space inward and outward drifts occur as a function of 
system size. In particular we notice that drift is a local quantity, at a given system size and at fixed 
model parameters the drift may be inward for one starting point in strategy space, but outward in 
other regions. The range of starting points at which an outward drift is observed is reduced as the 
system size is increased. In infinite populations the drift is inward throughout strategy space, with 
x* = 1/2 a globally stable attractor. 

An analytical estimate for the boundary separating the regions with inward and outward drifts 



can be obtained setting A(n) = in Eq. (14). Re- arranging and ignoring sub- leading terms in higher 
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powers of N l l 2 one finds that a reversal of local drift occurs when 

-= 1 ±^=, (15) 
N 2 ^/2N 

i.e. when (n/N — 1/2) 2 = 1/(2N), establishing a typical distance D = X/(2N) from the deterministic 
fixed point. Eq. ( |15[ ) thus indicates the point in strategy space at which, for a given system size, the 
reversal of local drift occurs. Conversely for a given point x = n/N in strategy space the reversal of 
local drift is found at a population size N determined by iV = [V2(x — 1/2)] -2 . 

These results can be connected to the well established system-size expansion approach to evolu- 
tionary processes in finite populations. We here discuss an approach based on van Kampen's system 
size expansion, an alternative line of reasoning, using a Kramers-Moyal expansion is outlined in 
the Appendix. Following |van Kampen 1992 IBladon et al 2010j the following stochastic differential 
equation for fluctuations about the deterministic fixed points can be derived 

m = -\m+v(t). (i6) 

The variable £ is here defined as the re-scaled deviation 

from the deterministic fixed point, the noise variable n(t) is Gaussian and uncorrelated in time, 
specifically one obtains 

(V(t)v(t')) = ^5(t-t>) (18) 

from the van Kampen expansion. We will not report the details of the derivation of Eqs. ( 16|17|18 ), 
these follow the steps of |van Kampen 1992 IBladon et al 2010] . It is however worth noting that 
these equations apply only in the long-time limit, when the deterministic dynamics have reached a 
fixed point. The van Kampen expansion can also be executed for transients or deterministic systems 
with attractors other than fixed points [Bol and et al 2009] . but this is not relevant for the purposes 
of the present paper. 

Eqs. ( 17|18 ) define a linear Langevin equation, and it is hence straightforward to compute the 
variance of fluctuations in the stationary state, see e.g. [Risken 1996J for details. One finds that 
3£ (£,(t) 2 ) = — \ {i{t) 2 ) + i]\M an d hence (£(i) 2 ) = ^ at long times t. This defines a 'typical' 



asymptotic distance from the deterministic fixed point in perfect agreement with Eq. (15). We 
therefore conclude that the stochasticity of the dynamics induces a typical asymptotic distance 
D = 1/(2N) from the deterministic fixed point, and that an inward drift is observed for starting points 
in strategy space that are further away from the fixed point than this typical distance. An outward 
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FIG. 2: (Color online) Dependence of the direction of local drift in the Hawk-Dove game as a function of the 
starting point x = n/N and of the system size N of the finite population. An outward drift occurs in a region 



close to the deterministic fixed point x* = 1/2 (shaded area, results are obtained from Eq. (13)), and inward 
drift further away from the deterministic fixed point. The range of starting points at which an outward drift 
is observed is reduced as the system size is increased. In infinite populations the drift is inwards throughout 



strategy space, with x* — 1/2 a, globally stable attractor. Black lines show the estimate of Eq. (15) 



drift is found for starting points closer to the fixed point than the 'preferred' asymptotic distance, 
underlining the importance to think of drift as a local quantity. In [Claussen fc Traulsen 2008], 
IClaussen 2007| similar calculations were performed (in the context of different games), but a flat 
average over all possible initial conditions was then computed, resulting in a global notion of an 
inward or outward drift, depending solely on the system size N, but with no 'local' character in 
strategy space. While expressions for the local drift were of course contained in the calculations 
of [Claussen Hz Traulse n 2008, Cl aussen 2007| as intermediate steps, we believe that our results and 
analysis provide an interesting addition and further insight into the dynamics of drift reversal. 



IV. IMITATION DYNAMICS AND INTERNAL ABSORPTION IN ONE-DIMENSIONAL 

SYSTEMS 

We now turn to the second main topic of this paper, and consider a microscopic update process 
which admits absorption at an internal fixed point even for finite populations. This is not the case 
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in the previous section, where we have shown that finite systems under the local update rule instead 
tend to a stationary state with fluctuations of a typical magnitude, or distance from the fixed point, 
even if the corresponding deterministic dynamics have a stable fixed point within the interior of 
strategy space. 

Specifically we will now consider an imitation process. As before two players are drawn at random 
from the population at the beginning of each microscopic update step. If they are of identical types, 
nothing happens. If one player is of type A and the other one of type B, then the two players compare 
their fitnesses, /a and If the two fitnesses are equal, fA = fB, the player of type A converts into 
a player of type B with a rate proportional to v/2, and similarly B switches to A with rate v/2. The 
non-negative constant u is a model parameter, and denotes the rate with which 'neutral' transitions 
occur. If however one player has a strictly higher fitness than the other, then the player with the 
higher fitness remains unchanged. The player with the lower fitness adopts the higher-performing 
strategy with a rate proportional to the fitness difference, for example if /a > Ib, then player B will 
turn into a player of type A with a rate proportional to Ja — Jb- No transitions to strategies with 
strictly lower fitness are allowed. Specifically we will consider the process defined by the following 
transition rates: 

r+f n nN-nfv w7r A (n) -Tr B (n)\ 
rp— t \ nN-nfv W7r A (n) - ir B (n)\ 

0(-) is the step function, i.e. we have Q(y < 0) = 0, @(y > 0) = 1. The deterministic limit of this 
dynamics is obtained as 

x = T°°' + (x) — T°°~(x) 

= x(l - x) I -sgn(7r A « - ttbM) + - M > BK 1 \ , (20) 

where sgn(y < 0) = —1, sgn(y = 0) = and sgn(y > 0) = 1. Inserting the expressions for tta and ttb 
of the Hawk-Dove game (b = 1, c = 2) we have 

, / v . wl — 2x\ , . 

x = x(l-x) -sgn(l-2x) + -- . (21) 

\2 4 A7r max y 

We will here mostly focus on the case v = 0, which we will refer to this as strict imitation, or strict 
proportional imitation. In this case the choice of w and A7r max is mostly irrelevant, and equivalent 
to setting the time scale. Unless specified otherwise we will mostly use A-7r max = w = 1 in the 
following. The word 'proportional' indicates that adaptation towards better strategies occurs with 



a rate proportional to the improvement in fitness. As first shown in Helbing 1992 one recovers the 
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FIG. 3: (Color online) Internal absorption in the imitation dynamics of the Hawk-Dove game. Left: Trajec- 
tories for N = 100 players, started from different initial conditions. The horizontal line shows the internal 
absorbing state, n* /N = 51/100, as defined by Eq. Right: Average time to fixation at the internal fixed 
point in a population of N — 100 agents, started at point n < n* = 51 in strategy space (solid line are results 



from Eq. (23 1, interpolated to non-integer n for optical convenience, symbols are from simulations, averaged 



over 10000 independent runs). 



standard replicator dynamics for v = 0, see also |Hofbauer &, Sigmund 19 98, Gintis 2000J. Indeed 



Eq. (21) reduces to Eq. (11) for v = up to an overall re-scaling of time by a constant factor 1/2. 

Even though the local process discussed in the previous section and the strict imitation dynamics 
have identical deterministic limits the dynamical features of the respective stochastic processes in 
finite populations are quite different. This is illustrated in Fig. [3j where we show a set of trajectories 
obtained from the strict imitation dynamics {y = 0) for the Hawk-Dove game (b = 1, c = 2), started 
at different initial conditions. As seen in the figure all trajectories are eventually absorbed at the 
internal fixed point given by Eq. Q. 

This behaviour can be understood from considerations of local drift taking into account that at 
v = we have T + (n)T~ (n) = for all n £ {0, . . . , A^}, i.e only one of the two transition rates is 
ever non-zero for any given n. If for example n > N/2 + 1, then we have /a(^) < and hence 

T + (n) = 0. Excluding the trivial case n = N, which is an absorbing state itself, the resulting drift 
is then given by 



A(n) = T~{n) [D(n - 1) - D(n)] < 0, 



(22) 



with the last inequality holding as T (n) > and D(n) > D{n — 1) for n > A^/2 + 1. A similar 
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argument applies when < n < N/2 + 1. At the fixed point n* = N/2 + 1 we have fA( n *) = /b(^*)> 
and therefore T + (n*) = T~(n*) = 0, confirming that n* = N/2 + 1 is indeed an absorbing state. 
One therefore concludes that fixation at the internal attractor will occur for any non-trivial starting 
point (i.e. n/O, n / N), and provided that N is an even number (ensuring that n* = N/2 + 1 is 
an integer). 

To complete the picture it remains to compute the mean time to fixation, given a starting point 
n. Assume to this end that n < n* . Then T~{n) = 0, and the system will move to state n + 1 with 
the rate T + (n), reaching the state n + 1 after an average waiting time of l/(T + (n)). If n + 1 < n* 
the system has not yet reached absorption. A second exponential clock will start ticking, and the 
system will hop to state n + 2 after an average waiting period l/(T + (n + 1)). This procedure then 
iterates until the absorbing state n* is reached, and we conclude that the mean time to fixation for 
trajectories starting at n < n* is given by 

71*— 1 

k=n 

Similar results can be derived for starting points n > n* . These findings are in full consistency with 
the results of e.g. |Altroc k~&; Traulsen 2009b] (see e.g. Eqs. (7) and (8) of this reference), taking into 
account for example that T~(n) = for all n < n* , and that absorption occurs at the internal fixed 
point with probability one. Comparison with simulations confirms the validity of our calculations, 
see the right panel of Fig. [3} As a technical detail we point out that the time scale used in the 



figure is obtained from that of Eq. (23) via the re-scaling t = t/N. This is to ensure a well defined 



thermodynamic limit, N — > oo, for example one may notice that the sum in (23) contains an O(N) 
number of terms, so that normalisation as just described is appropriate. 



V. TWO-PLAYER SYMMETRIC GAMES WITH CYCLIC INTERACTION 



A. Internal absorption 



We will now turn to a different class of games, and consider symmetric two-players games with 
three pure strategies and cyclic interaction. Specifically we will investigate a generalisation of the 
well-known rock-paper-scissors (RPS) game, defined by the 3x3 payoff matrix 



/ 



A 



1 -s 
-s 1 

1 -s 



(24) 
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where we will generally consider the case s > 0. Two players of identical species (strategies) will 
score zero payoff when they play each other. Strategy 1 will 'beat' strategy two (receiving a payoff 
of unity), but lose against strategy three, and similar for all cyclic permutations. The loser's payoff 
is — s < 0. The central point of the 2-dimensional strategy simplex, at which all three species are 
equally populated (xi = 1/3 for i = 1, 2, 3) is known to be a stable fixed point of the corresponding 
replicator equation, provided s < 1 jGintis 20001 IClaussen & Traulsen 20081 iMobilia 2010j . At s = 1, 
corresponding to the standard RPS game, the centre is a neutrally stable fixed point, and the 
replicator dynamics admits an integral of motion, with periodic orbits about the centre fixed point. 
At s > 1 finally the central fixed point is unstable, and deterministic trajectories approach the 
boundary of the simplex in a periodic manner. In absence of mutation, the case we will consider 
here, the three corner points of the simplex (x = (1, 0, 0), (0, 1, 0), (0, 0, 1)) are fixed points for any 
value of s. 

The strict imitation dynamics in finite population is then defined by the following transition rates 
Tir+j = 44 U Att - ) e fa( p ) -**(")). i,J€ {W}, i^j. (25) 

\ ITltlX / 

We here use A7r max = 1 + s and w = 1. 

As shown in the left panel of Fig. |4| this dynamics admits absorption at the internal fixed point. 
At the centre of the strategy simplex all three strategies are equally abundant, and by symmetry their 



fitnesses are equal. Hence all transition rates in Eq. (25) vanish. This is the case for any value of s, 
however as seen in the right panel of Fig. [4] fixation at the centre occurs with non-zero probability 
only at s < 1. To understand this in more detail, it is helpful to keep in mind that the system 
can either fixate at the centre (corresponding to a mixed strategy profile) or at one of the corners 
of the strategy simplex (pure strategies). Due to stochastic fluctuations fixation at one of the four 
attractors will occur eventually in finite systems. This is confirmed in simulations, detailed analysis 
of the data in the right panel of Fig. [4] shows that all trajectories fixate eventually. Generally the 
stochastic dynamics will be governed by two contributions, first the flow of the deterministic limit 
about which the finite system can be thought to fluctuate, and secondly stochastic drift due to finite- 
size fluctuations. Stochasticity tends to drive the system to the boundaries of the simplex, and can 
hence cause fixation in one of the pure strategies (once the system hits one of the edges of the strategy 
simplex, fixation into one of the corners occurs). The results of Fig. [4] can then be understood as 
follows: 



1. At s > 1 both the deterministic flow and the stochastic fluctuations drive the system towards 
the exterior of the simplex, and fixation occurs at one of the pure strategies. 
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FIG. 4: (Color online) Left: Trajectory obtained for the generalised RPS game under strict proportional 
imitation (N — 99, s — 0.5). The the three curves show the concentrations of the three strategies as a 
function of time, convergence to the internal attractors is reached at about t = 600. Right: Circles indicate 
the probability to fixate at one of the corners of the strategy simplex (pure strategies) if started from a 
random initial condition (N = 999), squares show the probability of absorption at the central attractor (mixed 
strategy). Results are from simulations (1000 samples, run until fixation), and shown as a function of the 
model parameter s. 

2. At s < 1 the deterministic flow pulls the system towards the centre of the strategy simplex, 
where fixation occurs, if the centre is reached. Stochastic fluctuations however provide a 
tendency to drift outwards, and provided one of the edges of the simplex is reached fixation 
occurs at a pure strategy. The deterministic pull towards the centre will be strong for values 
of s much smaller than one, and fixation at the centre is likely. For values of s close to one, 
the deterministic fixed point becomes less and less stable, and fluctuations more influential. 
The system fixates at the borders of the simplex with increasing probability, as seen in Fig. 
[4] (right panel), notice in particular the non-zero probability of fixation at a pure strategy for 
values of s just below one. 

Measurements of the probability with which the system fixates at the boundary for fixed values of 
s < 1, and at varying system sizes confirm that external absorption (i.e. fixation at one of the pure 
strategies) is indeed a stochastic effect induced by fluctuations. Fig. [5] shows that the rate with 
which the system fixates at the edges of the simplex scales as iV _1 at large values of N. 

This can be further understood from Fig. [6j where we show what the most likely point of fixation 
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FIG. 5: (Color online) Probability that fixation occurs at one of the corners of the strategy simplex in the 
generalise rock-paper-scissors game under strict proportional imitation dynamics [y = 0). Results are shown 
as a function of the size N of the population. Symbols represent simulations, averaged over 10000 — 40000 
samples (depending on s), started from random initial conditions. The solid lines has a scaling of 1/N. Inset: 
Average time to fixation at the boundary (open symbols) and the centre (filled symbols) for s = 0.5 

is for different starting points of the dynamics. If started in the central region of the strategy simplex 
(indicated by grey symbols) it is more likely that the system will fixate at the centre than at any 
of the three pure strategy absorbing states. For starting points closer to the edges of the simplex, 
fixation is likely to occur at one of the corners, as indicated by the coloured symbols. The left and 
right panels compare the basins of attraction of the internal absorbing state for different system 
sizes. As seen in the figure the domain from which the central attractor is reached grows as the 
system size is increased, and essentially spans the bulk of the two-dimensional strategy simplex for 
sufficiently large systems. Fixation at the edges only occurs for starting points at the periphery. This 
leads to the conclusion that the results of Fig. [5] may potentially be interpreted as an effect of bulk 
versus periphery. A starting point in the central region (which has a size 0{N 2 )) favours fixation at 
the centre, a start in the peripheral region (which has a size 0{N)) predominantly leads fixation at 
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FIG. 6: (Color online) Fixation in the generalised rock-paper-scissors game at s = 0.75 under strict adaption 
dynamics. The figure shows results from simulations indicating where the system is most likely to fixate 
given a starting position. Red symbols for example indicate fixation in the upper corner of the simplex. If 
started from the grey area near the centre the system is found to most likely fixate at the internal fixed point 
(1/3, 1/3, 1/3). System sizes are N = 33 (left) and N = 42 (right). Simulations are run for in excess of 10 4 
independent samples per data point. 

one of the corners. At large ./V and started from a random initial condition fixation occurs with a 
probability scaling with the relative areas of the peripheral and central basins of attraction, leading 
to a probability of external fixation scaling as 1/N. As a final remark we report that the average 
time to fixation at the interior fixed points (averaged over all initial conditions) is logarithmic in time 
(see inset of Fig. [5]). 



B. Drift reversal 



We will now briefly turn to a discussion of local drift reversal in the generalised rock-papers- 
scissors game. As pointed out above such phenomena can only be expected when the considered 
dynamics has a stable attracting interior fixed point in the deterministic limit. Also we need to 
exclude internal absorption. We therefore focus on the above local process, and the generalised RPS 



game, Eq. (24), with s < 1. 

The limiting replicator equations for the degrees of freedom x±,X2 and X3 = 1 — x\ — X2, de- 
noting the concentrations of the three strategies, are then known to admit the Liapunov function 
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FIG. 7: (Color online) Drift reversal in generalised RPS: The figure shows time series H(t) — H* from sim- 
ulations of the local process at two different system sizes (s = 0.5). In both cases the dynamics were once 
started from a fixed initial condition close to the central deterministic fixed point (increasing curves) , and once 
from a point closer to one of the edges of the simplex (decreasing curves). Trajectories which fixate during 
the course of the simulation up to t = 10 3 are excluded from the statistics, simulation data is from in excess 
of 300 non-fixated runs. As seen in the figure the initial drift is away from the fixed point in the former case, 
and towards the fixed point in the latter. The dashed lines show the asymptotic value of H — H* as predicted 



by Eq. (321 



jGintis 20001 ITraulsen et al 2005] 

H = -xix 2 (l - x\ - x 2 ). (26) 

Specifically one has H < under the replicator dynamics (s < 1), and the deterministic dynamics 
converges to the central fixed point (x^xJ^Xg) = (1/3,1/3,1/3), corresponding to H* = —1/27. 
In [Claussen &: Traulsen 2008] the occurrence of drift reversal was reported for stochastic dynam- 
ics for this game in finite populations. Specifically, random initial conditions in the strategy sim- 
plex were considered, and it was shown that different microscopic processes tend to lead to an 
increase of H in small populations (drift away from the central fixed point), and to decreasing 
values of H in large populations (drift towards the fixed point). The 'critical' population size, at 
which the drift reverses sign, can be computed within an expansion in the inverse system size, see 
[Claussen fc Traulsen 20081 IClaussen 2007] . Crucially however, drift reversal was considered only 
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FIG. 8: (Color online) Drift reversal in a generalised RPS population (s = 0.5) with N — 99 individuals, 
subject to the local update process. Yellow symbols near the centre indicate the region in strategy space at 
which the drift has an outward direction, darker symbols in the outer region indicate an inward drift. These 



were obtained from numerically evaluating the sign of the drift, using an analogue of Eq. ( 12 1. The solid black 



line indicates points with the value H — H predicted by Eq. ( 32 ) . Small deviations of the yellow region from 



the inside of the perimeter defined by H = H are due to finite-size effects. 

as a global phenomenon, averaged over all possible initial conditions in strategy space. The main 
objective of this section is to refine this analysis, and to stress that drift and its reversal are again 
local phenomena. 

This can be observed for example in Fig. [7j where we show that the initial drift is towards the 
fixed point (decreasing curves in the figure) , when the dynamics is started sufficiently far away from 
the centre of the simplex, but that a drift away from the centre (increasing time series of H) occurs 
for starting points close to the center. The simulation results in Fig. [7] suggest that for any given 
population size ./V the system chooses an asymptotic value, H, of H, and that drift always occurs 
towards this preferred value of H. Generally H will depend on the system size. Thus if, at a given 
value of N, the initial condition is such that the system is further away from the fixed point than 
indicated by H, an inward drift occurs. Similarly, for starting points too close to the centre, an 
outward drift, together with an increase of H is found, until H is reached. One should here stress 
that states with H = H are of course not absorbing, but that statements such as 'the system will 
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reach a value of H = H\ instead only hold statistically speaking, and when averaged over realisations 
of the stochastic dynamics. 

Analytical progress can here be made within a system-size expansion, which allows one to describe 
the fluctuations about the deterministic fixed point. Again we do not discuss the detailed mathe- 
matics, as these are available in the literature, but report only the final result, consisting of the the 
following Langevin equations for fluctuations in the x\ and x 2 components of the dynamics (fluctua- 
tions in the X3 component then follow trivially due to overall conservation of particle number) . One 
has 

■ w ( a . 1 + s, \ 

A7r max \3 3 J 

6 = ^-(-^-^) + -». <*> 



where are the fluctuations of the x-component, more precisely one has rii/N = x* + ^/viV for 
i = 1,2. The variables 771 (t) and 772 (£) denote zero-average Gaussian white noise with the following 
covariance properties 

(771 (iMO) = (m(t)m(t')) = 2 -5(t-t') (28) 
{m(t)m{t')) = -\s{t-t'), (29) 



as obtained from the van Kampen expansion. The linearity of Eqs. (27) allows further progress and 



in particular the long-time limit of the following quantities can be computed 

= <£i(*)£i(t)> > *22(t) = {Ut)Ut)) , ff 12 (t) = (721 (t) = (Zi(t)Ut)) , (30) 

see [Risken 1996] for further details. Denoting these asymptotic values by <7n,cx22 and o\2 respec- 
tively, one then has 

]7 -<( 1 ' : + 7f)(^ + 7f)( 1 - i: -^-7f-7f)>- (31) 



where ^ and ^ 2 are zero-average Gaussian random variables with statistics {d^j) = a ij f° r *> 3 = 1; 2, 
and where (• • •) denotes an average over £j and From this one finds 

^-^ = ^[^11+^22+^12] (32) 

This estimate of the stationary value of H — H* , accurate for large, but finite N, compares well with 
simulations, see Fig. [7j and confirms the picture discussed above. Depending on the system size the 



dynamics tends to a stationary value H — H* = 0(N ), see Eq. (32). Interpreting H — H* as a 



'distance' from the central deterministic fixed point, the dynamics is thus driven towards a state of a 
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preferred distance from the central point. If started from closer to the fixed point than this distance 
then the initial drift is outwards, if started from a point outside the asymptotic perimeter set by 
H = H, then the drift is inwards. This is illustrated in Fig. [8j 

As a final technical and more subtle point we would like to add that the corners of the strategy 
simplex still remain absorbing states, as we are working in the absence of mutation. One should 
hence expect that any trajectory of the system in finite populations will fixate on a pure strategy 
eventually. While this may at first seem to be in contradiction with the results just discussed, it is 
important to keep in mind that this type of fixation occurs on an exponentially slow time scale for 
s < 1, see e.g. |Mobilia~20 10]. This fixation regime is not captured by the van Kampen expansion. 
Instead the expansion addresses a regime in which a stationary Gaussian distribution is assumed, and 
where fixation at the edges of the simplex can be discarded. For the comparison of the theoretical 
predictions obtained from the van Kampen expansion against numerical data, we have therefore 
excluded trajectories which fixate in our simulations (see Fig. Y7n. 



In this section we will now consider a final example, and move on to a more complex asymmetric 
two-population game, Darwin's so-called 'Battle of the Sexes', also known as the 'Matching Pennies' 
game. Here there are two separate populations, each of size N, and labelled 1 and 2 for simplicity. 
Players in population 1 ('male') only play the game with players in population 2 ('female'), and vice 
versa. The payoff of a pure strategy in the male population is hence determined by the composition 
of the female population and vice versa. Payoffs can then be encoded in bi-matrix form as follows: 



The pure strategies in population 1 are labelled A\ and B\, the pure strategies in population 2 are 
referred to as A 2 and B 2 . The first element in the brackets for each of the four cases is the payoff 
of the male player, the second the payoff of the female. In simple terms a male >l-player prefers to 
interact with a female -B-player rather than a female A, and a male B preferably interacts with a 
female A (anti-coordination). Female players take the opposite view, a female A receives a higher 
payoff when meeting a male A as compared to meeting a male B, and similarly a female B prefers 
a male B over a male A (coordination). Of course the labelling 'male' and 'female' is not meant to 
have any direct real-world interpretation, the game would be identical with these labels interchanged. 



VI. IMITATION DYNAMICS IN TWO-PLAYER 2x2 GAMES 



A 2 



B 2 




(33) 
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FIG. 9: (Color online) Battle of the Sexes: Panel a) shows an illustration of the deterministic flow, and the 
five fixed points. Panel b) shows the region near the central fixed point, arrows indicate the transitions that 
are possible under the strict imitation dynamics. The central fixed point can never be reached. The arrows 
into and out of the central square have been colored in pairs. The two arrows of any given color will give a 
net outflow taking into account that transition rates scale in proportion to ratios of the type m(N — m)/N 2 
for example. 

We denote the number of male A players by n G {0, . . . , N} and the number of female A players by 
m G {0, . . . , N}. The payoff structure is then described by fitness functions denned as follows 

(l), \ m N — m (i) , . 
7ry(m) = -— + , TT y B '{m) = -vry(m), 

„£>(„)_»_ „£>(„) _ -„«(„). (34) 

Strict imitation dynamics occurs within the two populations, for example two male players may be 
chosen for potential adaptation, compare their fitnesses and then one of them may change their 
strategy to adopt that of the other male. This leads to the following transition rates 

T<».+ („, m) _ (f ^"'"l;^"""' ) 8 (iff M - „g'(m) 

r a,,- ( , m) _ ^(f ' g)( l-5 W )e(4''W-4''W 
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FIG. 10: (Color online) Fixation in the battle of the sexes under strict imitation dynamics. Colors indicate 
the absorbing state that is most likely to be reached if the dynamics is started from a given initial condition. 
Simulations arc for N = 20, 100001 realisations. The central fixed point at (n, m) = (10, 10) is left blank. 

The resulting flow in the deterministic replicator limit is shown in Fig. [9] (panel a) , and has the 
well known cyclic motion with neutrally stable cycles about the central fixed point at (1/2,1/2). 
While this point is an absorbing state under the strict imitation dynamics in finite populations, Eqs. 



(35), simulations reveal that fixation never occurs at the centre, but that instead all trajectories are 



absorbed at one of the four corners of strategy space eventually. This is illustrated in Fig. 10 , where 
we indicate the most likely point of absorption for all possible initial conditions of the imitation 
dynamics for N = 21. The absence of internal absorption in this game can be understood from 
a closer inspection of the imitation dynamics close to the central fixed point (see Fig. [9] panel b). 
In this figure we show the central fixed point (centre circle), and the surrounding states in finite 
populations (the population size is assumed to be even, so that the central fixed point is within the 
allowed strategy space). The arrows indicate transitions from one state to another that are possible 
under imitation, if no arrow is present the corresponding transition is not admissible. As seen in the 
figure the central fixed point cannot be reached from any of the neighbouring states, hence internal 
absorption can not occur. A closer analysis shows that there is indeed a net flux away from the 
region of the central fixed point, see the caption of Fig. [9) 
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VII. CONCLUSIONS 

In summary we have studied a strict imitation dynamics in games played by finite populations of 
players. It is here important to stress that strict imitation is at best a stylised dynamical process, and 
in the light of recent experiments [Traulsen et al 2010] may well be ruled out for real-word adaptation 
and learning of players. Still this dynamics has interesting dynamical features, as underlined by 
the early studies of Helbing |Hclbing 1992 1 , linking imitation to replicator dynamics. We have here 
shown that proportional imitation can give rise to what we call internal absorption, a novel dynamical 
phenomenon in which dynamic arrest occurs at an internal point in strategy space, corresponding to 
the stable attractor of the deterministic replicator dynamics. This effect occurs in simple symmetric 
2x2 games, provided the payoff matrix is chosen such that the replicator equations have a stable 
internal fixed point. We have also verified the occurrence of internal absorption in generalise rock- 
paper-scissors games, again the stability of the internal fixed point under the replicator dynamics 
appears to be a necessary condition. In the Battle of the Sexes, where the replicator equations have 
neutrally stable cycles, no absorption at the internal fixed point is found. 

The second main contribution of this paper is to analyse in detail the phenomenon of drift reversal, 
first reported in (Claussen &: Traulsen 20081 IClaussen 2007] . We have shown that the reversal of drift 
is in fact a local phenomenon, dependent on position in strategy space. Provided its deterministic 
limiting dynamics has a stable internal fixed point, a given microscopic update rule will select a 
'typical' distance from this fixed point at which it operates asymptotically. This distance will depend 
on the population size, and will be smaller for large populations than for small ones. If started 
closer to the fixed point than the asymptotic distance, the system will have a tendency to move away 
from the fixed point (outward drift), if started further away from the fixed point than the preferred 
distance, an inward drift will be observed. The precise domains of inward and outward drift can 
be obtained analytically within a system-size expansion, valid for large, but finite populations. In 
this limit we have also provided an analytical estimate for the typical asymptotic distance from the 
deterministic fixed point. 
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Appendix 

The point in strategy space at which drift reversal occurs for the one-dimensional Hawk-Dove ex- 
ample can also be obtained from a Kramers- Moyal expansion of the master equation (|3]). As shown in 
[Traulsen et al 20051 ITraulsen et al 2006] this formalism results in a multiplicative stochastic process 
of the form 

dx = F(x)dt + *^=W, (36) 



where dW denotes the increment of a standard Wiener process. One has F{x) = T°° ,+ (x) — T°°' (x) 
and G(x) = \jT°°> + {x) + T°°~ (x). From this one then finds (respecting the rules of Ito calculus) 

dD = D'(x)dx + ^D"(x)(dx) 2 (37) 
for the time evolution of the distance D from the interior fixed point. This then gives 
(dD(x)) = {D™)'(x)F{x)dt +1 ^(D™T{x)(G{x)) 2 dW 2 

= l(D™)'(x)F{x) + ^(D™)"(x){G(x)f)dt, (38) 

i.e. 

| (D\x) = (D°°Y(x)F(x) + (39) 

The notation d(D\x) /dt here indicates the expected change in distance per unit time at time t 
conditioned to trajectories starting at position x at t. The stationary point, d(D\x s ) /dt = is 
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therefore given by the condition (D 00 )' (x s )F(x s ) = — (D 00 )" (x s ) G ^n • ^ or ^ ne Euclidean distance 



2N 

D°°(x) = (x — x*) 2 this simplifies to 2(x s — x*)F(x a ) = , and finally assuming that x s — x* 

is of order iV -1 / 2 , we self-consistently find 



x '~ x ' = ± ]]~Wf^)- < 40 » 

For the case of the Hawk-Dove game with local dynamics we find F'(x*) = —1/4 and G(x*) 2 = 1/4 
so that we have 



— Wfif- <41) 



reproducing the result of Eq. (15). 



